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A System for Processing and Decoding MPEG Compatible 
Data and Internet Information 

This is a non-provisional application of provisional 
A P pri, C T9T 7 . Senal 60/ ° 43 ' 539 b * *• Lo^n et a' filed 14 

Field of the Invention 

This invention is related to the field of digital sienal 

of r °MPEG 8, T PartiC " lar,y '° Pr0CeSSin * - d *«X 

of MPEG compatible data and Internet information. 

Background of the Invention 

' 5 r„ . H °T e en,ertainment Sterns which combine Personal 
Computer and television functions (PC/TV systems), are 
mcreastnglv becoming, generic. User interactive, multip>e source 
and multiple destination communication devices. Such systems are 
required to communicate in different data! formats between 
2 0 multiple locations for a variety of applications in response to Use" 
requests. For example, a PC/TV system may receive data from 
r satellite or terrestrial sources comprising High Definition 
" Television (HDTV)- broadcasts;- Multi-point Microwave Distribut on 
1 2S (MMDS) broadcasts and- Digital Video Broadcasts (DVB)! 

25 PCTV system may also receive and transmit data via telephone 

- !™\ CM ' ial <** cabk TV) and from both 

remote and local sources such as Digital Video Disk (DVD), CDROM, 
VHS and Digital VHS (DVHS™) t y pe players, , PCs, and many othe 
types of sources. , . < 

3 0 A number of problems arise in developing such a 

generic PC/TV entertainment system. Specifically, problems arise 
in supporting communication between multiple data sources and 
m processing data from the multiple sources. A problem is also 
presented by the need to devise a User interface for such a system 

3 5 that supports complex User interactive tasks whilst providing *\ 
simple command interface suitable for the general; public. A 
PC/TV system User interface, for example, should.allow a User to 
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view a . selected program and permit User operation of functions 
such as Email, telephone, Internet access, fax and video-phone 
functions. Such applications require communication between a 
PC/TV unit and a variety of both remote sources e.g. a satellite 
5 service provider, and local sources e.g. a DVD storage device. 
Further, a PC/TV needs to process and decode data in different 
data formats from different devices and display . received data to 
the - User. These problems, are addressed by a system according to 
the present invention. i v . :i; ^ :o % ~ : ; .-<. 

10 ' ' .r:.- ■ vV; ■ '■ 

Summary of the Invention 

The inventors have; hereby recognized : that a Program 
Guide type of User interface advantageously provides a simple, 

1 5 -easy to . .use jnterfa.ce ,fpr] User operalibh of functions such as Email, 

telephone, Vlnte-rnet access, i; fax> : homev 'control- '' ahd ; vi r de6-phbne 
functions. The use of a Program Guidg for -^b-^fi&ttibirs-^also 
advantageously provides a single User interface for User operation 
_ of multiple^ devices and: associated 1 functions. - ; 
2 9rj.3u.; : \Th^;fi«weators.. jbavc;i ; ftfrthiar recognized that it is 
desirable for ; a video- i decoder system to " ; be- capable" 'bf : decoding 
, input ; yideo data^ ] encoded in ua: ^plurality r > of. encoding- formats ' 3 such 
as MPEG compatible and Internet compatible data- fbrrnats. 
Further, that it is desirable for a decoder to be capable of 

2 5 ^displaying . ; the- (iecbded ..- data derived from the different encoding ' 

formats e.g. decoded MPEG: vMeo -program ! ^data a'rid' 'web £age 
Internet data as a. -composite video image. -" ! • - 

A video decoder system decodes input video data 
encoded in ; a plurality of encoding formats: The decoder includes a 

3 0 first processor for decoding MPEG compatible data 'identified by a 

, first data identifier to. provide a first- decoded video output. The 
decoder also includes a second processor for decoding data 
encoded in an Internet compatible data format identified by a 
second data identifier to provide a second decoded video output. A 
3 5, display, processor formats . the first hand second decoded video 
-outputs for display as a composite video image and the proportion 
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of the video image contributed by the first and second decoded 
outputs is variable. 

• • • . • .Brief Description of the Drawings 

In the drawing: 

Figure 1 shows a home entertainment decoder system, 
according to the invention, for communicating with a plurality- of 
sources and for processing program guide information " ; an d 
1 0 program content data. 

Figure 2 shows an exemplary Program Guide display 
format according-' to; v the r invention. y- : o "1 

1 5 - 1 : Figure ... 3 shows -.-a;, data T formal according^ to the 

; invention, for packetizing- Internet data for transmission within an 
MPEG compatible datastream. ■ v. a • . 

. Fi « ure 4 shows a flowchart, according. -"to' the invention, 
20 for receiving, program guide, : information: ;including : User selection^ 
, options , and processing the received, information to • form h a 
,< program guide- ^supporting i User-t seIfecta^le--eoirfmWnic£t ! io^- 
. functions. ; N ,(..,•; s , r , .. i* :>;•-;•'. t; .3 

2 5 ; n . , Figure , 5 shows a flowchart; : according to the invention, ; 
;i for a process to form a program guide in a' video decoder from 

information received from a plurality of . sources. 

, Figure 6 shows a flowchart, according to the invention, 

3 0 for a process to ^automatically acquire and collate program guide 

information from a plurality of sources and to form a program 
guide for display. - : 

Figure 7 shows a black and white representation of an 
3 5 exemplary color Program Guide display , format, according to. the 
invention, showing menu- options permitting a User to collate 
programs by source and color attribute. 
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■ - Figure 8 shows a flowchart, according to the invention, 

for a process to acquire information requested by a video receiver 
User via a selection made from options on" a displayed program 

5 guide. 

Figure 9 shows a flowchart, according to the invention, 
for a process to decode input 1 Internet information and 
compressed video data and to form a composite video output for 

1 6 display. «. ;• 

' " Detailed Description of the Drawings 

<: ' '■ ;Vi '" : Figiire' 1 shows a hbme^ entertainmerit decoder system, 
1 5 according to the' invention, .for communicating with multiple 
" Sbtirces; to acquire program '.' f 'gu;de' ' mfor'mat'iqri ' ''and program 
5 content data. The decoder system 'forms a program guide for 
1 display from information acquired and collated automatically from 
multiple sources. A User may initiate Email, telephone, Internet. 

2 0 access, fax and video-phone functions via selection of options from 

the ' displayed program- guide. The" decoder system adaptively 
" decodes a datastxearri encoded to . the" MPEG standard incorporating 

' "compressed program content ^.*^"..' an .?.^t.- I .^' n ?^---..^? ta 
representative "of, for example', Hypertext Mark-up. Language r 

2 5' (HTML). 'The'- 'decoded data is formatted either as .a composite 

video image or as separate video images for display. 
' ' ; - : The MPEG2 (Moving Pictures Expert Group) image 
" encoding standard," hereinafter referred tq as the "MPEG standard" 
: is ^comprised of a system encoding section (IS O/IEC 1 3 8 1 8r 1 , 1 0th 

3 0 Juh^ 1994) and a video encoding^ section (ISO/IEC 13818-2, 20th 

" January 1995), hereinafter referred to as the "MPEG systems 
standard" and "MPEG video standard" respectively! , 

Although the disclosed system is described in the 
'"context"' of a' 1 system 'for '.receiving ; an MPEG compatible signal, it is 
3 5 'exemplary only, Ttie principles of the invention may be applied to 
^systems in which the " types of transmission, channels and 
communication protocols may vary, or to systems in which the 
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coding type may vary. Such systems may include, for example, 
non-MPEG compatible systems, involving other types of encoded 
daiastreams and other methods of conveying . Program Guide 
information. Further, although, the disclosed system is described as 
5 processing broadcast programs, this is exemplary only. The . term 
'program' is used to represent any form of data such as telephone 
messages, computer programs, Internet, data or other 
communications, for example. 

In overview, in the video ^receiver system of Figure 1, 
10 a carrier modulated with video data is received from a broadcast 
satellite by antenna 10 and processed by unit 15. The resultant 
digital output signal is demodulated, by demodulator 20 and 
decoded and error corrected by decoder 30. In this exemplary 
embodiment, the demodulated and decoded output from unit 30 is 
1 ? in the ? ori ? of an MPEG co^atiW^.^transport siream. containing 
MPEG compressed video data and Internet data encoded in HTML. 
Video data encoded to the MPEG standard , is' in the form of a 
packetized datastream which typically includes the data',con^iit,.of 
_ many program channels (e.g.. content ^ corresponding to cable 
2 0 television channels 1-1 25) . " . " ! " 

, The MPEG compatible transport stream is provided 'to^ 

processor 25 which is responsive to commands from "remote 
control unit 125. s Processor "25 . co^uniciates'j; with_. other '"data 
sources including storage, device 90 and "either Internet \data 

2 5 server 83 or : 'Internet connection service 87 (e.g. /Amekca7pn r 

line™). Program guide information is ^acquired and cojlated 
automatically from, the input transport stream ^ and / storage device 
90 and either server 83 or connection service ibV. The acquired 
information is merged by processor 25 to form a prpgiram guide 

3 0 for display incorporating selection options i allowing a User to 

initiate Email, telephone, Internet access, fax and video-phone 
functions among others, for example. The program \ guide' for 
display is also formed to include non-displayed information from 
multiple sources for identifying and assembling individual data 
3 5 packets that constitute, the programs that are. selected for User 
viewing or listening. This information is used in recovering the 
content of selected programs from multiple sources. - ^ 
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Processor 25 adaptively decompresses the input 
compressed video data and decodes the HTML Internet data' to 
provide a formatted composite video image for display on unit 75. 
Processor 25 also provides encoded data outputs for storage on 
5 storage medium 105 via storage device 90, and provides encoded 
data to other devices (not : shown to-- - 'simplify drawing) via modem 
80 and telephone .lines. - ~ 

, A vi*ieo receiver* user initiates functions including 
Email, telephone,; - Internet : access; ofax : r and video-phone functions 
10 and cr yiewing: : and v storage of: selected- - programs via* 'r selection of' 
rj options : rf?o.m a ; theC displayed ^progrim "guide: < r : A-- : -User selects a 
: djspteyesd - option svi th -av cursor tisirig > remdte control unit 125 . 
T: Cpmipa$ds, from; remote control- a25^ar£ p&ssed 'via interface : 120 
: Jo .controller, ' 1.1 5if/:withiil>:.prdcesSb^« 2Sr^Unit : 115" controls 54 ' the 

1 5,, operation of the^elements^ of processor 25 ^ancf* resp'oniis to remote 
3 fcontrql ^unife f 25 commands; u data anil control 
fj-sigpafc ^us.?^ GanCroller/ 115 controls the function^ of individual 
^elTexnejats withinJiprocessorc 25: by ^settitt^ control- -register values 
Ar^ithin^thesp elements^ witfiS^coitrQl bus^ C. Processor Tt 25 ' al so 

2 Orsiipports ; ;the storage and ;iretrieVa£^f- data ^frbm Storage " riie&ium 

o^5Kyia ^storage devieeil9Q. /Storage device 90^ is a DVD type device 
ande:rm^diumv:;105^vcomprises,^ a ^ttiulti-diSfcs stack:- unit containing a 
plurality of disks in this exemplary embodiment. : v 

- c:H ;^n:-Gonsideririgi^Figurte i *iif ^detail^a' carrier modulated 

2 5, with\ video- dataxreceived by * aiitenha 10, J is converted - to digital 

-fdrm-rand 'processed by input processor 15. Processor 15 r 4ncludes 
t radios frequency (RF)i tuner and^ interriiediate 1 frequency (IF) mixer 
-and.r a;mplification- stages for down-converting the input video 
signal - to , ;a; lower frequency band. Processor 15 also includes an 

3 Q:^alog to b digital jconvetter: for ^ digitizing the down-converted 

signal to produce a signal suitable for- further processing. The 
: .resultant digital output signal is demodulated by demodulator 20 
r and decoded aiid rerror-cotrected -by decoder 30. The> output from 
^ decoder 30 is i further - processed by unit 45 of processor 25. " r 
3 5 - i ; t o The: data provided to Unit 45 is in the form of an MPEG 
compliant : packetized transport datastream as defined in MPEG 
systems standard sectioji 2.4 and includes program guide 



WO 98/47286 



PCT/US97/17036 



7 

.information and the data, content of one or more program 
channels. Processor 25 acquires and collates program guide 
information automatically, from the input transport stream and 
storage device 90 and either Internet data: server 83 or Internet 
5 connection service 87 (e.g.. .America On-line™),: The - individual 
..packets that comprise either particular, program channel content, 
or Program Guide information, are identified" by their Packet 
... Identifiers (PIDs) contained within : header information. 

-. • • In c response to icontrol signal C, unit- 45 selects either 
10 the transport, stream from unit- 30; or in a playback mode, a 
datastream retrieved, from storage device 90 ; via < store interface 
J 95 : 1 ? norrna i. non-playback operation, individual/ packets. ' that 
comprise program ^ guide . information are identified by -unit -45'- in 
the ; transport stream , frprn unit 3.0 and Assembled using -PIDs that 

1 : a .f e : Pjedetermined a,nd. . stored, in internal memory of - controller 
: ;.; 115 - rUnit 45 ; matches., -the ., PIDs of. incoming packets', in the 
; • ^^ a < 1S P 0 • rt it5 lat ^t?■eam^ from decoder 30; with vPID • values pre-loaded 
, in control, .registers . wjthin unit:; 45> by -controller 1 15. Therefore, 
c satellite - broadcast program guide (SPG), information -packets? 4n- the 

2 9-:5 r , an . s P or t; : stream i a re identified by unit 45 without additional TPID"' 

information. -Controller 115 acquires -a full SPG by : 'accessing" "and 
, assembling the- SPG , packets that are. identified . and ' captured ; by 

unit 45. ■-' '••; .■ . ■ , ' r c '. .- 

i . . Ti 16 PIP /: information ;that : ^enables , controller 115, in 

2 5 conjunction with ; unit 45, to identify data .■■ packets! that : comprise 
- in 4?Yii dual P r PS ram s is termed a ; program or . channel map: The 
=t ^9fi ra : m ma P associates; , PIDs with : individual packetized 

.datastreams that constitute a program and is part of- the SPG. The 
SPG may also contain additional information, e.g. packet assembly 

3 0 information, supporting the recovery of program I guide data from 

the input, transport stream. .-„• : . . 

Controller 115 also configures unit 45 using Control 
signal C . to select the data packets comprising :the DVD program 
guide (DPG) information derived, from DVD . storage device 90 via 
3 5 interface 95, Unit 45 matches the PIDs of -incoming jackets in the: 
packet datastream from interface 95 r with PID, -values pre-loaded 
in control registers within unit 45 by controller. 115. Controller 
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115 acquires a full DPG by accessing and assembling the DPG 
packets that are identified and captured by unit 45. 

A similar process is followed in acquiring Internet 
program guide (IPG) information from Internet server 83 or from 

5 Internet connection service 87 by telephone line communication 
via modem 80. In acquiring the IPG, data identifiers analogous to 
or the same as the PIDs used in acquiring the SPG or DPG are 

ffl: - 7 ihatched -"wittf identifiers pre-loaded 'by'- controller 115. The 
matching function may be performed either within unit 80 or by 

1 0 controller ' 115. The process " for acquiring program guide 
' information is explained in gf eater detail in the description of 

Figures 4-6/' * ; - J " -'-^ 4 " ^ " ^ _ ^ 

1 ; The" SPG; : DPG arid IPtJ information Acquired by 

controller ' 115 is rnerged by processor 25 to forni a program guide 
1 5 for display incorporating Selection Options allowing a User to 
initiate Email, telephone, Internet ac9ess, fax and video-phone 
'functions. In merging the ; data, the acquired SPG, DPG and IPG 
information is into twor levels of "hierarchy termed theme 

0 and^ to^ic and redundant program guide information is eliminated. rf 

2 0- A theme, for example, may include categories sucK as movies, ~ 
\; : spbrtsv : 'Veather, :i: arts, documentaries, he\vs, etc. A topic is a 

7 category within a theme, for example, a ; movie theme category 
may have topics including, coiriedy, thriller,^ horror, science fiction, 
Vomaiice, adult entertainment,: etc. T>iffefenf or additional levels of. 

2 5 hierarchy ' may be used in collating the acquired SPG,~ DPG/ and TPG 
; information involving criteria such as viewef ' age suitability, the 

period setting e.g. contemporary of historical, reviewer ratings etc. 
~ * A number of problems occur in collating the SPG, DPG 

and IPG information. The SPG, DPG and IPG information, L, being 

3 0 from 1 different sources, may categorize programs by, different 

themes and topics and may* use a different sorting hierarchy. 
Further, the SPG, DPG and IPG data may use conflicting, redundant 
" or incompatible PIDs or other data identifiers for identifying 
individual data elements that comprise particular program 
3 5 : charinfcl or guide content. Controller 115 creates a master set of 
• : themes and topics mapping the 'SPG, DPG and IPG information to a 
corresponding category in the master set. For example, the master 
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set may contain an encompassing topic-theme of 'movies-comedy' 
to which is mapped both, a) the DPG topic-theme of 'movie- 
parodies', and b) the IPG Internet topic-theme of 'movie-synopsis 
index' (an index of web pages). In collating the guide information, 
5 controller 1 15 determines the topic and theme categories of the 
data in the SPG, DPG and IPG from . pre-determined theme-topic 
codes. The SPG, DPG and IPG theme and topic codes together with 
equivalence mapping , information for. converting codes, , of /.one 
service provider, to those of another, are acquired , from the SPG 'or 
10 User defined data input to processor 25 or are pre- stored, in 
internal memory 1 within controller 115; Controller -. 115 - assembles 
the collated SPG, DPG and IPG packets according to the categories 
defined by the master set of themes and .topics. The assembled 
plated ^PG, DPG and IPG data is stored by controller 115 in 

1 5" internal memory for use in forming a composite program guide", for 

display. "_. ■ - - • _ ''\ 

D *~ OT ^° Uf ? } 15 also forms MPEG compatible non- 
displayed information,: associated with the displayed composite 
program guide. The non-displayed . information," includes ; composite 
20 Program- map, conditional access and network .information , This 
non-displayed information is required to support " identification, 
assembly and decoding of the packets comprising the composite 
: pr6gram g uide and ; the " associated • program content. The 
conditional access information governs access to programs based 

2 5 upon User entitlement. The network information defines ^physical 

network parameters such" as satellite transmission channel 
frequencies and transponder channels, for example. 
'•; Controller 115 forms composite program map 

information for the composite program guide that maps, the SPG, 

3 0 DPG and IPG themes and topics to the master set of themes and 

topics. The composite program map also associates data identifiers 
(e.g. PID values) with individual packetized datastreams that 
constitute' '.programs listed in the composite program guide. The 
composite program map information may also include a dedicated 
3 5 indicator to denote that a . particular program has associated 
Internet web; page information ..available. In forming, composite 
program map information, controller 115 examines the SPG DPG 
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and IPG information for data identifier omissions, redundancies 
and conflicts. Controller 115 re-numbers existing data identifiers 
and creates new data identifiers as^ required to produce MPEG 
compatible program map information for both the collated 
5 program guide information and the associated program content. 

Controller 115 also forms MPEG compatible conditional 
access information and network information from corresponding 
information received in the SPG, DPG and IPG data. The conditional 
access information includes data . such as encryption codes that 
1 0 govern access to programs that are dependent* upon user 
.entitlement. The network information defines (physical network 
parameters, such as satellite transmission * channel frequencies, 
transponder channels, ^nd -internet access parameters associated 
; with the, compqsite. T program guide ford display. - The composite 

1 5 program map; conditional access] ami ; network information 

supports assembly, decoding and display - of the program content 
f available from : the satellite, storage^ and Internet server sources as 
uWg 11 R as r the r cpmppsite program <-guide, ! ; i ^ u ;>:.v i ^ 
'«-."ryn$ H Control incudes; additional linking data in 

2 0 the QQnTrdisplayed cpmppsitq : program , mapv- conditional access T and 

^network .information^ : The, : additional : Unkings data? supports the 
;P$pX selectable functions r ^uchj as /Email^^ldephone, Internet lacceiss, 
t fax r u video-phone, /channel - -tuning, program* : recording and - home 
r jqontrol r functions. r . Home control ■ functions ^mayo ;support User 

2 5 ppnirol . o^.Mghtang air conditioning,- heating; and ^burglar ilarm ? 

operation, fpj example, n The ^ additional linking; data associates 
access r data,, both ^ with data identifiers (:e*g. o PIDs) of requested 
4<Ua, ^a^nd^ with User selectable t ;menu options in the displayed 
composite program guide. TSie access, data ? . includes ; identification 

3 0 codes, v communication -protocol ; codes, conditional access - codes, 

Internet access codes e.g/URL, codes for; web page and Email access 
as well as telephone and fax numbers. The access data enables 
processor 25 to communicate with : external devices. 

In order to generate the composite program guide for 
3 5 display, controller 115 retrieves the assembled and collated SPG, 
PpG^ and IPG packet data previously stored in- internal memory. 
Controller 115 both : : converts the retrieved data to pixel data and 
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formats the converted data for storage in a pixel representative 
memory map contained in display processor 70. In addition, 
r controller 115 creates User selectable menu options (for User 
selection of Email, fax etc.) in pixel data form for incorporation in 
5 the unit 70 pixel memory map. Controller 115 -stores the 
formatted SPG, DPG arid IPG pixel data together with the 
generated option pixel data in the unit 70 pixel map to form the 
composite j program guide for display: ^ 
The composite program guide, combining the SPG, DPG 
10 and IPG. data, and User selectable options, is formatted to be 1 
compatible with the . linking data in the non-displayed program 
. guide information , For/ example;' a User selectable option iri the 
-.displayed guide (e.g. for ; a telephone function')"- fc- associated,' i: 66th ' 
with access data: for: :the function (e.g. a telephone number), 1 arid 
15 with data identifiers. (eig-PIDs for data packets of ^transmitted jind ? 
received. -.telephone messages). ; : : - 

-..> Display processor -70 converts ' the " stored arid 
formatted program guide data into Soriveritiorial luminance arid 
.chrominance components. -The luminance and 'chrominance 
2 0 components, together; with timing -data- provided by- controller f l5, 
.are processed by processor i 70 in a conventional manner to form 
« an NTSG signal -for.; (display on :NTSG compatible image- reproduction 
'device, 75 e.g. . a television or video monitor. Processor 70 may- in 
another; embodiment, provide RGB ^output data for display by : an 

2 5, RGB compatible reproduction, .device or a high definition' "television'. 

;:: Controller: 115, in conjunction with processor 55; forms 

• an .MPEG compatible -datastream -of compressed" video and audio 
data , containing .the composite program guide 1 and c program 
; content. Processor 55 separates system information including 

3 0 timing, error and synchronization information from the 

datastream; from unit 45 and diverts it for use by controller 1 15 
Controller 1 15 applies the system information in controlling 
processor 55 to provide re-synchronized, collated program and 
composite program guide data together with synchronization and 
3 5 error indication, information 1 for - use in subsequent program 
content data decompression by decoder 65. The composite 
program guide data provided by processor 55 combines the SPG, 
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DPG and IPG data and User selectable options in a manner 
compatible with the program map linking information as 
previously described. The program content data provided by 
processor 55 is decompressed by MPEG decoder 65 to provide 
5 video pixel data and audio output data to" processor 70. Display 
processor 70 processes the vidbo pixel data and audio output data 
to be suitable for reproduction by unit 75 as previously described. 
' Figur6 2 shows an exemplary displayed composite 

' program guide. Menu features 833-850 show program services 
10' and functions. Menu features 833 and 835 show program channels 
which may be invoked ahd displayed upon highlighting of the 
""'cMrtespoTidito^ menu feature using a User; selection qontrol such as 
^remote cohtror directed 'cursor; 1 If a User 1 highlights feature 833 
with the cursor, i.e.* selects menu option 833, for example, 

1 5 processor 25 tunes to television channel 107 (A&E™) and" displays 

it ori unit 75 rising the composite non-displayed program gurde 
infbrniktion to' determine the function activated and assopiated 
tuning ^^ Inforiiiatioh' (channel frequencies etc.);' Similarly, if a User 
highlights feature 837, /processor 25/ tunes to /receive radio channel, 

2 0° FM : 13 for audio output on uhit 75 , Upon highlighting * features 

•840-.850, correspbridin| to / Email, telephone, fax and DVD 
'rectfrd/playback functions respectively, additional function 
specific menus become displayed providing further. User selection 
bptipns\ Such i functiph/ specific m^ may provide a User with the 

2 5" ability "td ^dit, store and delete/ access information' ^ including, for 

example/ telephone numbers, Email addresses, Internet addresses, 
credit card Aumbers and additional conditional access, data. , 

;? t; - If at User highlights . irfenu features 860-870 specific 
' function tasks are initiated; * Upon a; User highlighting feature 860, 

3 0 for example, a further menu is displayed prompting the User to 

filler an Email message to be sent to John. Smith at his home via 
: ah internet Email address retrieved from the composite non- 
displayed program guide information. Similarly, in response to a 
User highlighting /feature 865, for example, controller 115 (Figure 
3 5 1) I retrieves Jean's home telephone number from internal memory 
Using the composite^ non-displayed* program guide information. 
Further, controller 11*5 establishes telephone communication, via 
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modem 80, with Jean at her home by dialing Jean's home 
telephone number. Also, upon a . User highlighting feature ,870, for 
example, the movie 'Star Wars™- is re-played from DVD unit 90 
(Figure 1). The communication between . processor 25. and. i DVD 
5 unit 90 is established with access data, retrieved from . memory 
within controller 115 using the composite non-displayed program 
guide information. : ' 

In addition, the ^program guide may include one or 
more icons, such as icon 873 /(Figure, 2) depicting a home, fpr 
1 0 example, to indicate whether Internet information such as a 'home 
page' is available for a particular program.. Further icon 873/ ! in 
Figure 2 may change color e.g. from white to green tp .indicate 
Internet information is available for 5 a/, particular "program. 
Alternatively, icon 873 may be a pop-up icon which .appears as ; an 
15 indication that Internet information is , ayaUable / for .a ~, particular 
program. The displayed program guide may be ; formed ! to enable • a 
User to request Internet information on a particular program, by 
: Highlighting, the program followed by highlighting the ppp-up^cpn, 
for example. Home page , icons that/ are present when . internet 
2 0 information is .available may Vsp , 'lqcatj5d;>ithin. or. adja'Unt to 
; program icons, e.g. icon 859. Further, the home" page * icon .may 
change color upon User selection, of the infp>ma^Ipn''.' : ^y 
highlighting the home page icon, / ' ' ' 

; Ali %?.H^. ^t«raetJinfoi:matidn may ;~ be retrieved upon 

2 5 highlighting a web page* icon such as icon .857, in Figure 2,.\his is , 

exemplary only; ketrieyai of Internet information may also be 
initiated in a variety of other ways. A user may retrieve available 
Internet, information relating to a program, by highlighting a 
program icon such as 'Seinfeld™' using icon 859 followed by 

3 0 highlighting "Internet -icon "810, .for 'example.' 

Upon . User . selection of a composite program guide 
function, controller 115 (Figure 1) determines and applies the 
access data associated with both request and response operations 
involved in the selected function. Controller 1 15 determines the 
3 5 access data required fpr the request and .response operations' frpm, 
the previously formed non-displayed composite/ program map, 
conditional access and network information. " 
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Processor 25 acquires information, such as web page 
data, in response to a User request, in a number of ways, 
including/ for example: 

5 a) In a first configuration, processor 25 accesses web 

page data that is continuously transmitted in the satellite 
broadcast received : from decoder 30, 

: ^ 7 b) In a second cohhguration 25~ acquires 

1 O'wdb' /page ^d^ta that is 'transmitted in th6 satellite 1 broadcast 
^received 1 from decoder 3 & in respbSse X6 £n information request to 
i a : serviee^ piro viBer %ia 1 'telephone^ 'line, ; Itiuf ! 

* t? r " ™ - • ^ 3 c) Ti Irr "a ^ird ^cohlfigufatidiil processor t>5 both requests^ 

1 5 J mfdrniatio'n &nd v receives resp^tfsfe " J : information t foil'*" the "same 

: eommunicatiott link^e\g.^ t>n * tlftr telephone lini: to server 83. 

- First - Icb^TigOT^ian of !Pr6cesis6T 7 25 r 1 ^ u ; l • 1 

2 0 nou ' 2 ^ ;r ( :> irt ^he r first a dohf^ selected by a 

tJteer^vi^ the" ^i^layed"progriart gliidet v is cdntinubusiy transmitted 
within " y tfre <: tf ansf>'6rt u;: ^atastreanf * 'irtjftif ^ td 1 ' 1 pTbces'sbr 25 from 
'decider 30. In H this Cbhfi^^tion^ v the' 66 program map 

^inferfnatiori- cbntains 3 predet^rfiiirifed : Pit) values" for ^ flie requested 
25 : web - pag6. Go&equehtly, did? requested^ web pige data is retrieved 
"'fibril the 1 input tralispbirt strfearti usittg the c'dmpdsite^ prbgram*' map 
^iMormkUon' 7 ' without' 4 the need L fbf controller 115 to communicate 
"with a service provider via modem' 80: In this configuration, the 
^3b a Ihfdrniation available to a User is limited to the specific 

3 04iifbffi^ion 0 that a service " provider cbbtihuously transmits to 

processor 25 usirig dedicated transmission/ bandwidth. However, 
this : configuration is;siiftple and "offers quick web page access 
(since the' wel> data tftay bb- stored locally at the encoder by the 
service provider). This cbnfiguratioh also minimizes the service 
3 5 provider btirderi of data acquisition and eliminates the need for 
processor 25 * to communicate Web page request information to a 
service provider. 
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....... , The web page information, encoded in HTML™ and 

Java™, is packetized and formatted in an encoder by the service 
provider for incorporation into the MPEG transport - datastream for 
transmission. Figure 3 shows a data format, according to the 
5 invention, for packetizing Internet data for transmission within an 
MPEG compatible datastream. A sequence of . HTML™/J a va™ web" 
pages (705-720 of Figure 3) is formed' by . the,, encoder .^nto". ^on- 
compressed packets with ancillary information and HTML™/Java T1 ^ 
data (725). The ancillary information includes .PID values, a packet 
1 0 count value, and individual Web page size (e.g~ number of -bytes ,pr 
image pixel size), The ancillary Information i may, also - include:, -a 
flag to indicate that a, particular transport ■ packet 
page information; a start code; a web page "identification code; an 
error correction code and.,- a . timing .parameter, . suitable for 

1 5 synchrpnizing' web page Internet , information, with an- associated. 

video or. audio- program. The; : web. page flag .may y be ,used,'to 
facilitate identification and separation of the web page 
information from the MPEG .compressed video data, r Further* one 
or more elements of the web page ancillary data may be included 

2 0. within. .U^er definable elements .of. the MPEG, transport syntax, such- 

?7: tn * n - ans ,P^ t head ef. or. it may.be, included., along^with 
r the web page mfprmation K .in," the, MP0G . transport ..data payload. , 

- w, - ^ P'; o!^? s iii We ^ .Pfge information, including 0 a 
K pred _ ete? ™^5 d P R ^^ : ue ' : is : incorporated in the MPEG compatible 

2 5 transport datastream input to, .processor 25. (Figure 1\ .by . s the? 

service provider. The response cjiannel predetermined PID v.a^lue 
is also prestored in controller J 15 . .internal ■ memory ^ and 
incorporated in the composite program map. . JnforjnatipnV^ As'. V an 
alternative, the response data PID value and associated program 

3 0 map data may be encoded, for example, . in a Channel Information ; 

Table (CIT) in *Mz% definition television (HDTV) signal encoded 
according to the. r . Digital.. Television , Standard for HDTV 
Transmission of April" 12 1995,. prepared by., the/ United States 
Advanced Television Systems Committee (ATSC). In such .a case, 
3 5 one or more specific .pre-determined s program channels r jn ay , be 
designated to, -convey web page response information; Further, the 
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PID values of the program channels may be conveyed in a CIT in 
the composite program map information, for example. 

Controller 115 derives the predetermined PID value 
corresponding to the requested web page " response information 
5 from the stored composite program map information and loads it 
into control registers within unit '45. Unit 45 matches the PIDs of 
incoming packets of web page response data in the transport 
datasfream - from decoder 30 with the PID - Value pre-loaded in the 
unit 45 control* registers. The "packets with ' PID : values matching 
1 0 the -pre-determihed web response PID ■" value are "identified- as- web 1 
r-p&ge' jespbrise ^Uata &rid are K prbvided * by unit- 45 to a ; dedicated 

- buffer • in ^decoder '-55 : The- "dedicated ! buffer : in - transport decoder 
v- 55 3 holds ^h>spottsc ? daia'"to"-be^decoded by processor 60. ! ^ ; 

r u\j V. - Pro'c'esso'r 55 ; deriveV "system information^ including 

1 5 timirigV ! error ^nd synchronization ' information fro'rir the 

datastM^ '45 and' provides T it to ' 'cbtitroller ^1 15. 

Controllers 11 5 J -1 applies "the i - system inf ohriatrbn in directing 
r- 1 processor 55 * 4o ? provide re-syttehrbriized rt web '-'page 1 response data 
• from the' - dedicated = buffer in decoder 55 'to HTML™7Java™ 

2 0 processor i 60. Processor 60 :; decodes : the ftTME 1 "^ and Java™ encoded 
i^wgb 'response 'information r and pro'vides "web page image 

- representative' pixel data If or storage 1 l in ' } membry within coritMler 
1 15^- Although- pfocfessbr 60 "is shown* 'as a • separate processor in 
Figure r lv : it may ^alsb *be ; 'implemeft^ software* of 

2 5 -eohtrbllerl l*5v Processor '60. comprises 1 the 1 processing functions'; of 

a 5 special Web browser, including jPEd r decbinpressibH aiid Java 
decoding v -such/ ' as^ involved* in the : operation^ of Netscape 
Navigatbf™, for example^ - - 

Program content data e.g." a 2 '"User selected television 

3 0 program channel, Within the MPEG compatible transport 
^■datastream 1 from 7 decoder 30, is alsb -decoded into -pixel 

representative data by processor 25 in addition to the web page 
response data. For 4his purposef controller 115 determines the PID 
value of the selected television channel program ' from the 
3 5 -composite program map information arid loads ' it into control 
registers within unit 45. Unit 45 matches the PIDs of incoming 
packets of : selected television channel program data in the 
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transport datastream from decoder 30 with the PID value pre- 
loaded in the unit 45 control registers. The packets with PID 
:. -values.-: matching the -pre-determined selected television channel 
program PID value are identified and provided by unit 45 to a 
5 dedicated application, buffer in decoder 55. 

As ..previously described for the- web page response 
data, processor .55 derives system information including timing, 
error and .synchronization information; from the datastream i from 
unit 45 and provides it to. controller 11-5. Controller 11.5. applies 
1 0 the system information in, directing ..processor 55 to .provide re-; 
synchronized program channel content data from the application 
buffer in. decoder 55- to : MPEG decoder 65. The MPEG compatible 
program channel, content data is , decompressed by MPEG i decoder 
j- 65 to Provide video pixel . data, and audio output data to display 
15 processor 70. , The video pixel, data from decoder 65 is stored in, a 
pixel representative, memory contained ? in display processor -70 
; , ( Controller, .115 retrieves . the web page image- pixel : data 

from internal memory and , formats, it for storage as '= an -overlay! in 
the - ,pixel representative, . memory, f contained r , in- - display - - proeess'or 
20 70.„As, such *, merged display. ..combining . the . web page response) 
\ i data an , d program ; channel content- is , formed within -the -pixel 
, , memory of. , processor ^ 70.,, The^ relative -proportions and sizes of ^the 
.; web . page image , and channel content, image . within the . resultant 
- composite image, -formed :j in . the. processor 70 pixel memory* may 
25 be varied by controller, 115, in response. . to -internal programmed 
, ; .instruction , or external, -^s'^ User, command. , The .relative 
. proportions, may be varied: so that either the z web, page image or 
channel content image contributes between 0 7 100% pf \the 
resultant composite image.. . 
30 In order to adjust the proportion of the composite 

image contributed by the web page data, contrpller 1 1 5 scales^ the 
web page image pixel data by a ^conventional interpolation 
process. Alternatively, the data may be scaled by a separate video 
processing integrated circuit operating in , conjunction . with 
3 5. controller 115. The resulting scaled web page image is stored 
within the processor 70 pixel , memory by; controller 115.. The 
stored web page image represents an, overlay image with .respect 
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to the program channel content image. In order to adjust the 
location of the web page image data within the composite image, 
-controller 115 determines the addresses corresponding to the 
desired memory locations from internal memory map data and 
5 stores the scaled web' page image data at the desired memory 
locations. As previously explained, display processor 70 converts 
the formatted composite ' image data into conventional luminance 
and chrominance components. Further, processor 70 processes the 
luminance and chrominance components, together with timing 

1 0 data provided by controller 115, in a conventional manner to form 

an NTS G signal for display on NTSC compatible image reproduction 
1 device 75; ■ - ' 1 y - r - ' : "' 

Second configuration of Processor 25 

is ^ : ; \ ~ ' : \ ; ; ; ■ ' * ; \ V' ~ ! 7 ; ' 

r " — ■ In the second configuration, processor 25 of Figure I 
acquires web page data that is transmitted in the satellite 
broadcast received 'from decoder 30 in "response to an information 
request to i service provider via a telephone line. 1 Upon a User 

2 0 requesting web page data, e.g. by highlighting web page icon 857 

in Figure 2, controller 115 (Figure 1) uses the request access data 
(telephone number, requested web page identifier 1 code (e.g; URL) 
and conditional access data) to establish communication with the 
satellite broadcast service provider.^ ' 

2 5 Prior to establishing communication with the service 

provider, controller 115 determines from 1 conditional" access 
information whether access to the requested web page 
information is authorized. Authorization may \ be determined 
directly from the program guide "conditional access information or 

3 0 indirectly from a User entitlement device such as a smart card 

(riot shown to simplify the drawing) within processor 25. If access 
is authtirized, controller 115 establishes telephone communication 
with the satellite broadcast service provider by dialing a 
telephone number via iriodem 80 and transmitting request 
3 5 information e.g. requested web page identifier code (such as a 
URL) and user entitlement code. The service provider incorporates 
the requested web page information in the transport datastream 
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transmitted to processor 25 via. antenna 10, processor 20 and 
decoder 30. Controller .115 uses the response access data (PIDs of 
the web page data, response program, channel number and 
transponder . number), to identify .and . assemble the web . page 
5 packets in , the transport stream , as., described for the ,, first 
configuration, of processor .25. ' 

; The PID values of the response web page information 
and corresponding program map information may be 
communicated between servipe provider and processor 25, in a 

1 0 yariety of ways and in a variety of- data formats. The PID values 

an j d . program ma P: in ?P rm ^ tion ma y be conveyed, for example,,, in 
accordance with proprietary or custom requirements of,- a 
particular system such as within MPEG user definable data 
elements or within non-MPEG . data such as ; within the vertical 
15 blanking interval (overscan region) of a broadcast'' '"television-. ' 
r signal. Further, the PID Values allotted to the web information 
niay also be transmitted to processor 25 in supplemental program 
£" id ^j nfo * mati(n V 4 s : an< ? ! ther alternative,. t the PID values , to be 
used for response data, may be - determined by a service provider 

2 0 from, .information, transmitted from processor . 25. . \ - 

Third configuration of Pr^y fS «nr ?V ' „ 

In a third configuration,, processor 25 both requests 

2 5 inf °* mat :! 0 " and ?" e ceives response information on the same 
- .communication link e.g. on the "telephone link to server 83 or 

connection service 87, via modem 80. In this configuration , 
controller 115 requests. , Internet web page information , by 
highlighting icon 857 . (Figure ., 2), for example. . Whereupon 

3 0 controller 115 (Figure . 1) uses request access data (telephone 

number and URL code and conditional access data) to . establish 
Internet access and request web page data . at the URL address via 
modem 80 and server 83. Controller 115 determines whether the 
requested Internet access is authorized from conditional access or 
35 smart card User entitlement t information as described in 
connection with the second configuration of processor 25 If 
authorized, controller. 115 \ establishes telephone communication 
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with a service provider via server 83 by dialing a telephone 
number using modem 80 and transmitting the Internet URL 
address code and user entitlement code, for example, as 
previously described. Controller 115 uses response access data 
5 (including Transmission Control Protocol/Internet Protocol 
(TCP/IP) identification data) determined from the composite 
-program map information to identify and assemble the web page 
TCP/IP ' packets containing the web page HTML and associated 
-JAVA v tfata (and other data Cgl 7 JPEG, GIF, TIF type data) in the 
10 response datastream from server 83~. * 

;; ? : /o : Th« response ••web page' TCP/IP packets are' buffered 
^ within modem' 80 ; "and - provided to" HTME™ : processor 60 "-under 
'command of J controlled 115: Processor ^60 'decodes the ifTML™ and 
1 Java™ encoded '- : web" ^espbhsfe" information' "arid provides w£b -page 
15 image repFeseftidtive 5 px^el data : for stbrage in memory" within 1 
controller 115. 'Controller il5 retrieves" the e web s ;page image pixel 
^ata from 1 internal -m^ciry n ahtl formats- it for storage as an 
r * overlay in the pixel representative 11 itiemory ; 'contained - in display 
• firocfessb : f 70 fdr teproduciion by as a 1 merged display, as 

2 6 °previ6usly described. M 3 this : configuration^ processor 25 supports 
full Internet atces^ via server ' 8 3' '-using' the displayed program 
giifd'e. For exainplef upon af User l hlglilign6ng- icb'h' : 810 followed by 
icon- 830 (Figure- 2) 'controller 115" (Figure 1) executes web page 
"broVser functfdns 'and displays' either ^a ' custom web page ;browser 

2 5 or a standard btbwser display such 1 as Netscape Navigator™ - 

^hrbUgh which 'full Internet ' access is available. >- 

■ i ' ^ ! Alternatively, in the third configuration, processor 25 
* '"may "support a limited Internet service. In this case, access to the 
^Internet network via server 83 i : is under the control of a service 

3 6 ' provider that allows only indirect Internet access. The service 

provider provides collated web page information for display 
^ compatible with the displayed' composite program guide. The 
service provider stores a limited number of web pages locally 
within server 83 that are accessible by processor 25 using the 
3 5 composite 5 program map information which provides an index to 
" the available Web pages and their corresponding predetermined 
1 PlBs.' Alternatively, the service provider J may alter the selection of 
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web. pages available from server 83 and periodically provide 
updated compatible supplemental program guide information to 
processor 25 Via the satellite broadcast datastream.: 

The supplemental program, guide, information enables 
5 processor 25 to form an updated displayed guide " together with 
compatible, program map and other access information enabling a 
User to access , the updated web information/. By ; storing, 4he^. web 
page Internet information locally in r server ,83. the . service 
provider retains control of the * Internet . content available; to "a User 
10 and is able to bill the User for , the/material provided^ A^pJ jlie, 
local storage of the web page information in server 83 provides a 
User , with faster, access to, the information., without , "the .full delay 
associated with, normal Internet access. Further,. .'the, burden on 
processor 25 of communicating information. requests ( to external 
15 . devices is minimized jn that only a limited; quantity" of Internet: : 
; .. information ; may , be , requested and processor r 25 need' , only 
. communicate with local servers : such as server . 8 3^ It is to; be noted 
that modem 80 may also be a wide /bandwidth i/ communication 
modem such as a cable .modem. :Jn. s which . fajse^processorVl^^y 
2 Q decode both web page data and broadcast vi^q^pTO^aj^^^^or,.,. 
display from .a cable program source via modem 80. '/.' / ~ ' - 

•:• Coi ^£ Ue f. x i I?: .cmploys. ; the method shown . jn Fi^re 4^; 
. t0 : produce . a ; displayed, prpgramr guide - by 5 processing ' and 
v formatting, received,., prQgr^;, gui^e. r .. infprmat^on ..'.that." '.ajjf^qdy 

2 5 includes User .selection menu options. . The process pf Figure /4 does - 

not require processor 25 to forifl : User : selection m^.pv^m^ 
may be employed by processor s 25 in its first configuration/for 
example. In step 205, following the start at step 200, ..prpsesspr .25 
receives a transport datastream from . decoder 30 .^aC'.iWprppr^tes 

3 0 User selectable menu options for display as part of the.idisplayed 

program guide. The User, selectable options support User initiation " 
of communication functions such, as Email, telephone," In tejnet 
access, fax, video-phone, channel . tuning, program recording and 
home control functions. 
35 The data repeived in step 205 . also inpludes additional 

linking data in the program guide information. The"/ additional ' ' 
linking data comprises, program . map, conditional access , and 
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network information and associates access data with the User 

selectable menu options in the displayed program guide. The 

access data includes, identification codes, communication protocol 

codes, conditional access codes, Internet access codes e.g. URL 

5 codes for ^veb page and EmaiF access as well as telephone and fax 

: numbers. The additional linking data- enables processor 25 to 

. assemble; decode aitti display 4he program content ' and program 

r gtiide -information r in the 1 transport streim^ The 

access data enabled processor '25 : tb- cbminunicate with external 

1 0 devices . and: includes ! di#fe?eht : -addfe^s^ : representative -codes 

^corresponding to addresses of differ In 

r.step i210v : controller (; 115 T ^usesK predetermined linking' data PID 

lvalues:- stored iif internal ^memSry " in : * directirig pioc^ssor 25 \ to 

^identify and ;capture the 7 linking - 'aha" ^access J data 7 frdtff the "received 

1 5 prografn guide ^ ^irifb relation. £ The' captured" linking ' data * is 

assembled, formatted and stored by controller 115 in internal 
. rxfemory fei step --215t Also, in 1 -step 2lft; additional linking data 
r such a& i Ittte*nefc> -or* Erii&il * a3'diksse§ 1 k>r v ~ telephone numbers fnay jbe 
0 eriteYed'-fey^ia -UsSJ-vFa— an ^'entfy'-^eVxc^ ''siiefc as a'" remote control 

2 O^uhit ?an$ the : additidnal- dala^ls incorporated in c the received linking 

: ill : oz r< The - ■ pro-am -^guide^ j inftfnriaticrir containing ttie a °&ier 
^ s^le£Mble ^ofetiohSi 1 i^ ^oraiatted fbr r ^i^pla?y 1 by Control!^ " tl^in 
conjunction with display ^rdfaessor' 7 (r within processor 25 in step 

2 5 220, ^ as " previously^ dd&>Kb"ed in- -coiMwUbii^'-'Wit&" Figure 1 V In Step* 

225, display pfoc^ssbr 70 jo fccM verts ^ and formatted 

program -guide data n intd conventional luminance and chfoitimance 
components. The luminance and chrbmiriance cdm^onien ts , 
-together :; with timing data provided by controller 115, are 

3 0 - processed by - processor 70- for display on" reproduction device 75'. 

- The process of Figure^ 4* terminates ;: at Step X 230V ' 
. c v: r : ; Controller 115 J <Figure I) employs the mieth^d shown 
ih Figure . 5 to produce a' program ^guide in a video "decoder from 
information received from a~ plurality : of sources. The "formed 
3 5 guide -supports User .selectable communication functions e.g. 
Internet access, Email, fdx etc. . In sltep 305, following the start at 
step 300, controller \\5- acquires satellite program guide (SPG) 
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information from the satellite, broadcast transport . stream received 
.by processor 25 from decoder 30, In step 310, controller 115 
..acquires DVD program guide (DPG) information from storage 
device 90 via interface .95. Controller. J15„ acquires; collates and 
5 formats the SPG and DPG by . accessing and assembling respective 
data packets to produce SPG and DPG representative, pixel, data, for 
storage as a composite program guide in the ..unit 7.0 pixel :map v , as 
previously described in connection with Figure 1 ,- The- -SPG: and: J)PG 
information includes linking and^ access data in ,the form, !of 
10 program map, conditional access and network information,^ In 
f ° ther configurations,, processor 25 applies , the - principles -detailed 
. - .herein to form a . .composite,, program . guide.;; including; .prqgram 
, t S uide ; information ; derived from a ~ source, -rsuch as a , terrestrial 
broadcast, source, .a ; cable- broadcast ;source, .computer source,' a 

1 5 radio transmission source and. a spurce v accessed^ via . telephone 
■ . , v lines-. .' ;.. A " . ; 

/ : In , ste P 3 15 > ;:: controller r l : l 5 creates -Use?- selectable 
_ . menu options for display (for ^User, selection;, o& t fafcpfufe access, 
,.Email, fax etc.) in pixel data form, for ..incorporation .in. tiie> uni^?70 

2 0 ;P ixel memory map. In step. .,315,." controller , 11 5} ;i also ; ., creates jiUseC 

determinable menu options that enable a c : pser :; jo , customize, : his 
, program guide , tq, include , an - icon ; that . performs s f a : specific User 
requested function, , fqr example, a,^ser : ,.ma. X squire , an [ icon, f that 
accesses a specific . Internet site or-.results , in a display of . all 

25 received messages. Controller, 1 1 5, responds, to ; cpmmandsL en tered 
, Yi *. a data entr y d ? vice ; ; e -g- a /remote ■• control • unit c* keyboard; in 
creating the User determinable menu , option in accordance . with 
programmed instructions. ......... . - 

• Controller 115,, in step 320, examines the SPG .and , p£G 

3 0 information for data identifier (e.g. PED) .-omissions.. and,- conflict^ In 
step 325, controller 1 15 ; renumbers, -existing; data : identifiers . .and 
allocates new data identifiers as . required to produce compatible 
program map information for both the collated program guide 
information and . the associated, program content. In step 330, 

3 5 controller 115 forms composite program map,- conditional access. 

and network information for the. composite. -program r Vuide4n r the' 
manner described in connection with Figure 1. - *\.. 
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The composite program map is formed to enable 
processor 25 to perform the communication functions selectable 
via the composite program guide. For this purpose, the composite 
program map associates a communication function PID and menu 
5 option (e*g. web page; Email, fax icon) with the different address 
representative codes corresponding to the addresses of the 
different ; information sources. In step 330, controller 115 also 
incorporates User entered : data into the composite program map, 
conditional access and network information. This data rnay include 
1 0 items such as an Email address, a telephone/fax number, ' an 
•Internet '-web ^ page link address, a credit card number etc. The 
"0^' ertt!erS this data to processor 25 vii a device such as a remote 
conttol ? <rr° keyboard? ^ontrbller 115, iii step 335; formats" the SPG 
r - and c 'BPG a pixel 1 data together with ttfeP rtienu* option pixel data in the 

1 5 -unit : 10 pixel map cr^alled ; in ; step 315 to fdrni 1 ^ composite 

~ : r r : ThV composite program giiiHe inf donation 1 containing 
^Che -User seiebilble^'ii^ • is ^ formatted fbf display in step 

'^$8; •and^butptit- in 1 ste£ : 345 as 'previously* described*" in connection 

2 0°witti°ftgiire L Thfc process of Figure 5 terminates at step^350. 

— W< *>* : - (*<yntrollter :i i 1 ^fi^ure l^^pib^s prbcess of ^Figure 
i^Ho idirtonliatidariy : ^quire a knd cdilafe ^^ainl" guide information 

-frbnr^ pli&ility : 6f -$6mc£s l to fbnrf 'a *' composite; ' program ? ^uidfe ' for 
&is£lay?*In : ste^-^05r^olic^irig : tti& l itm yti. ' } Step : ~406, : - cbntroiie'r' ' 11 5^ 

2 5 Acquires £tiid Stores ' ^at^liit^^^b^nf" glMe (SP^) "rnforiSaition 

from the satellite " brbacica&t transport J strearii received *by 
pfbcessbr 25 from ^decoder 30. Controller 115,- in step 415, 
^airtbmatically niitiafes ^communication- "with" DVD storage device 90 
(Tu^^ vid 1 interface 95 ^ thfe" cbliimunic'ation is initialed upon 

3 0 ^bwer-iip of processor 25 in response to pre-stored hardware 

configuration iiifbrmatibn defining the peripheral devices that are 
attached J to processor 25 Vnd the associated communication 
protocols. This information may be Contained within the received 
program guide information,' or may be stored within processor 25 
3 5 at manufacture; or may be determined locally by processor 25 or 
may be determined through a combination of these methods. The 
local determination of hardware configuration by processor 25 
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may, be accomplished either by , peripheral - interrogation initiated 
by controller 1 15, or by User data entry of hardware configuration 
data. 

In alternative embodiments, cpmmunication may be 
5 automatically , initiated upon conditions such as, a) User . command 
via a program guide option, selection,, for example, b). peripheral 
interrogation initiated by. controller 115 on a periodic v;or 
intermittent . basis, or, c) . detection of a new or changed electrical 
connection of processor 25 to a peripheral , device such as DVD 
1 0 device 90. ■ . 

,. Controller 115 initiates ; communication with. D,VD. unit 
.i 90 : vi , a : interface - 95 usin ;g communication.' protocol, codes, 
.-.identification codes, , entitlement codes and: conditional ^ access 
^ odes retrieved from/the access . data within : the received, program 

1 5 guide, information. In step 420, controller. )l 5 ^ directs , processor ".25 

to acquire and assemble the DVD program guide (DPG) information 
derived from DVD storage device , 90 via interface 95 in the 
manner, . previously .described in .connection with .Figure. 1, In step 
425, . controller 1 15 collates the SPG and- DJ>G informatipn^by^ theme 

2 0 and ; topic by mapping £he r .jSPq c . and, J DPG r'infcimation.'.to^ 

qorresponding., theme and .topic categories' in - V; master set; also as 
. described in connection with^ Figure. VI Further, , in , step 425, ^ 
■ ConttoU.er ..ll5 collates .the SPG^and DPG information - by source ! ? and 

allocates a, different, .color attribute to programs and program - icons 

2 ? J TQm different- source^ In: .^addition, _ - controller 115 creates - menu 

options perrnitting a User, to sort by source, and cpjon .... ' r /./ . 

Figure 7 shows a black ,and.. white. jepresentatioaVpf,. a 
s color Program Guide display, format, -"according to the inyention, 
. showing menu options created by controller. 1 15 permitting, a. User 

3 0 to collate programs by source and color , attribute. Source, icpns 

905-925 are associated, with off-air broadcast, cable, radio, 
satellite and storage (e.g. DVD source. 90 r Figure, 1) respectively. 
.Source icons. 930-950 are associated, with, Email, telephone, 
videophone, fax and Internet function . sources respectively, Upon 
35 a User highlighting source icon 905*. for example, a .program ./guide,- 
is displayed showing offVair broadcast programs. „and associated 
icons. Similarly, upon a , User highlighting source icon.. 930.,' ".'for 
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example, a program guide is displayed showing Email icons via 
: which a User may activate menus for initiating composition and 
sending of Email messages and reading and deleting of messages 
received. ; 
5: Icon 900 is a "rainbow'* -'sort icon showing a spectrum 

of discrete color blocks represented by shades of gray in the black 
and white program ~ guide depiction of Figure 7. The color choices 
of icon 900 may alternatively be arranged as a continuous color 
spectrum or selected range of colors" in a single or a number of 
1 0 ^discrete color blocks. Upon a user highlighting ; a particular color 
within icon 900, programs from Ifce source attributed with that 
particular color are displayed/ For example^ assuming in step 425 
r controller 1 15 attributes the^colbr purple' to the DVD source device 
<; 90, then upon a uset highlighting 1 ihe color pu%>le ^ within icon 900, 

1 5 the movies available from DVD device 90 become displayed as a 

program guide. Alternatively, the" color collation scheme may be 
* used to identify programs by other criteria than source. Color may 

-be used as an attribute to identify programs by theme br topic, for 
r example^ in which Case a display of movies available for viewing 

2 0 Yrofn J ail sources may be identified by highlighting a red color 

- within' icon 900, for example. In addition, program guide data may 
collated by -attributes 'Other - than ?; cbibr;* : For "example, /programs- 
from different ' sources may be attributed with different types of 
shading of different icon Shapes and 'geometryS: * 1 

2 5 : . 1 v: Continuing with the process of Figure 6; in step 430, 

controller 115 forms a compbsite program guide for display from 
the collated SPG and DPG information in a manner similar to that 
' described in connection with Figure lV Controller 115 also, : in step 
425, creates associated User selectable menu options for display 

3 0 including the User selectable program guide options for sorting by 

source and color. The created menu optibns are incorporated by 
' controller 115 in step 430 ' in a compbsite program guide for 
display as exemplified in Figure 7. In step 435, controller 115 
formats the composite program guide information for display as 
3 5 previously described iri; ; connection with Figure 1: The process of 
Figure 6 terminates at step 440'. 



USDOOlD: <Wn SfM7PftfiA1 I > 



WO 98/47286 

>>^mv:q:?!j:vr.>T PCT/US97/17036 



27 

Controller 115 (Figure 1) employs the process of Figure 
8 to . acquire web page information requested by a video receiver 
User via a selection made from . options on a displayed program 
guide. In step 505, following the start at step 500, controller 115 
5 retrieves access, data in the .form of : a telephone number and- 
Internet URL code, for example, from memory in response to a 
User selection of a web page made -via a displayed program guide. 
The access data associated ; with the particular User /selection; is 
determined from a program guide , database .using program , .map 
10 information that, links all data associated > with -a particular' ; 
program or program, guide function. ; .......... r . ,. V. \ 

i '■ :: ■ In „ tyP 5 1.5 » j controller ; 1 15 directs processor >■ ; 25 - to 

. es J atjlish . a request communication link with server £ 83 by -dialing 
the retrieved telephone number using . modem lr 80. Also, in step 
15 5.15, processor 25 encodes and transmits additional access data's; 
such as the Internet URL code using, the protocol, required for 
. communication, on the request communication link. The required > 
k P TOtocoi is determined irorp the, program map . information, Jn, step 
, 520, controller 115 directs, processor 25 L to receive and decpde 'an 

2 0, input signal , received on a, response communication' link. Gontroller ..; 

115 . directs; processor .25 . in identifying and retrieving; the i 
response .weh page information, from^ the input signal in steps 525 4 
• and ; 530 r Different ; methods, of L retrieving response ] data \ from the 
input signal are ernplpyed. , by ;. prpcessor 25 ? in. steps 525 and 53JD. 
25 cT he selected method . depends, on, whether or not the response, 
communication link is the same link used , for the web ; page data 
, request, The methods used in Identifying and , retrieving ..the 
; response data are similar to those .previously described ,. s in 
connection ; with the. . three configurations of processor- 25. 

3 0 Controller .115 applies the. appropriate . configuration method in , 

identifying the response, web page data in the input signal, in step 
525. For, this purpose, controller 115 uses data identifiers 
determined from program map. information that associates the 
identifiers with the User program, guide selection. Controller . 115, 
3 5 in step 535, formats . the response , web page information for 
display and the Figure 8 process ends at step 540. ... 
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- Controller 115 (Figure 1) employs the process of -Figure 
9 for 'decoding both input Internet information and compressed 
video data to form a composite video output for display. Following 
the start at step 600, controller 115, in step 605, determines 
5 packet identifiers for- User requested Internet data and 
compressed image data from program map information. In step 
607, controller 115 determines whether access to the ~ compressed 
video data and the" requested Internet data is authorized from 
conditional access or smart card User entitlement information as 

1 0 previously described in connection with the third configuration of 

processor 25. 

If access is authorized, controller 115, in step 610, 
identifies the web page TCP/IP packets containing the web page 
data in a response datastream from server 83. Similarly, if access 
1 5 is authorized, controller 115, in step 615, identifies the 
compressed video data packets containing a User selected video 
program in an input transport stream from decoder 30 to 
processor 25. Controller 115 identifies the web page and 
compressed video data packets in a similar manner to that 

2 0 described in connection with Figure 1. 

In step 617, controller 115 directs processor 25 
(Figure 1) to decrypt Internet or compressed video data if it is 
encrypted. Controller 115 in step 620, directs processor 25 in 
decoding the identified Internet web page data using HTML™ 

2 5 processor 60. Controller 115 in step 625, directs processor 25 in 

decompressing the identified compressed video data using MPEG 
decoder 65. The resultant decoded web page data and 
decompressed video data is merged, formatted and displayed on 
unit 75 using display processor 70 under the direction of 

3 0 controller 115 in step 630. The process of Figure 9 terminates at 

step 640. 

The architecture of Figure 1 is not exclusive. Other 
architectures may be derived in accordance with the principles of 
the invention to accomplish the same objectives. Further, the 
3 5 functions of the elements of processor 25 of Figure 1 and the 
process steps of Figures 4-6, 8 and 9 may be implemented in 
whole or in part within the programmed instructions of a 
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microprocessor. In addition, the principles of the invention apply 
jo. any form of MPEG or non-MPEG compatible, electronic, program 
guide. . Further, the principles of ..the invention apply to 
communication on any form of .communication link including via 
cable, infra-red, radio, microwave, and via a computer network. In 
addition, it is to be... noted that the , f qrmatipn of . the- composite 
program guide as described herein may be performed at a. location 
remote from a video receiver and accessed, by the receiver via a 
communication . network. 
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CLAIMS 

1. Video decoder apparatus for decoding input video 
data encoded in a plurality of encoding formats, comprising: 
5 a first processor for decoding MPEG compatible data 

identified by a first data identifier to provide a first decoded 
video output; 

a second processor for decoding data encoded in an 
Internet comffatible -data format, ^ J saitf :; Internet' data being 
1 O' identified by 'a- second- data' identifier to provide a : second decoded 

video ~ butput; - -arid ;Tir -- rv ' ' "' ■ '" : " _ "*~ ' " 

a display processor for formatting said first' arid second 
a dbciided - video •^iftputs' ' for display as a : ^riiposife video image 

^wherein 

1 5 - aatasm »ai e ^Vop^rtoi^of -¥a»d video" lmige^-contftb^teci ' •fcy^slaid- 

^fifst -arid ; seebfrd decoded outputs 4s variabfei 1 " 

..=....- 2. Apparatus according to claim 1, wherein 

;'/qr r -j jn *u said-' -first : and second processors (decode data separated 

2 OMsirig^-said ■■. first arid? second' data 'identifiers frbrri s a single composite 

input datastream. 

' Vi i ■' ii . .(J i i i. r, : * ! \) j ».--v . .vi ~. „ - 

3. Apparatus according to claim 1%^ therein-'' •*»•"*-• 

•-. .g« ^ a^saitf 'first ^prbcessm^^ddcbrillressiBS ^MPEG compatible 

, t4 , r „.,.,. £T ..K .jr J*.r<~'. -t Hi. t : £ 

2 5 compressed data. * 1 '*" - 

o rjjgjj y/'CPi :« ' ^~> : -i J?\-» Oifi? M.TI.OC 0 

4. Apparatus ' according to" claim 1 l.whefein " 
'-.:.[■■, mhb vftf-g w-'-^eVond-'- = pirocesVdr ' • decodes "data formatted 

according -to the TCP7IP ; protocol.' - ~ ' 1 ' - ? 

3 0 : : ' r - — ,; > ' " - .... ■ 

/ r> r 5; Apparatus according to claim 4, wherein •'" \ 

i^I. said Iriternet'compafibie data format 1 mcludes 1 data 
encoded in at least one of a) HTML format, b) Java™ format, and c) 
ActiveX™ format. 

35 
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6. Apparatus according to claim 1, wherein 
said proportion of said video image contributed by 
v said first decoded output is variable between O. and 100%. 

5 7 - Apparatus according - to claim 1, wherein 

said display processor, stores ; &aid composite video 
image in a pixel memory. 

8 - A PPaf a tus according, to claim 1, wherein . M . ...-,< 
10 . , said display processor formats, said first. : and- seqond, 

decoded outputs as separate images within said, composite : video 
; image and . . . . '. :.„-,., . 

• s f id proportion of said ; composite video ; image 
contributed by said first and second decoded outputs is variable in 
1 5. response to at least., one of (a) .User selection, . .(b) .formatting data* 
received in said input video data,., and. . (c) \ pre r prog^mmed 
processor instruction. 



k ~ ; A method . for . decoding image representative input 

20 . K ideo ~ da ta; .encoded in a. plurality , of encoding;-, formats,, ^pmprisin^ < 
the steps of: 

identifying MPEG compatible first image'** 
representative, data;. . c . • . . f c _ . , , ,- 

3 :i ;-; ~ : Wcntifyjng o ^sepon4, L image • . representative data encoded 
2 5 in ah Internet compatible data format; _.,., K ' 

decoding said first image representative data using a 
first method to . provide^ a first decoded output;,,- — 4 >, 
.. . decoding said second . image , representative data using 
a second method to provide a second decoded output; and , % 
30 formatting said first and second decoded outputs for, : 

display as a composite . video image . wherein the .proportion of said 
video image contributed by said first and second ; decoded outputs 
. is variable. . 
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10. A method according to claim 9, further including 

the step of: 

separating said first and second image representative 
data from a single composite input datastream. 

5 ■ ' : ' ' ' * " ' 

11. Apparatus according to claim 9, including the step 

of " ',* : v-- r - - 

determining whether a User is authorized to access 
said first image representative data and" wherein' 
10 said step of decoding said first image representative 

data occurs in response to said authorization. 

12. Apparatus according to claim 11, including the step 

5 of ■ ■ ■ 1 - v : " : . ; 

15 : > decrypting said first image representative data in 

response to said User authorization. ' 1 

■■: - :13,' : A ; method -accdrdmg :v to : ciaim :r 9, Including the steps 

of ' / L . \ ' ' _ 

2 0 - receiving program guide information arid 

^ u selecting web page information for display from said 

received program guide information. ; ' * " 

14. A method according to claim 9, including the step 

2 5 of 

receiving an index of web page information and 
including the step of 

selecting said second image representative data from 

said index. 

30 

15. A method according to claim 9, including the step 

of 

decompressing MPEG compatible compressed first 
image representative data to provide said first decoded output. 
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16. A method according to claim 9, including the step 
of .. . \, , . • : , r 

decoding second image representative data formatted 
5 according to the TCP/IP protocol. 

17. A method according to claim 9, including the step 

of , ^ Vw . , ^ : r - : : . r . _ . v . , ' " ' 

varying . : . said , proportion., of - said video image 
1 0 contributed by said first decoded, output. . , , . 

18. A method according to claim 9, including the step 

decoding second image representative data formatted 
1 5 in at least one of a) HTML format, b) Java™ .format, and c) , 
ActiveX™ format. 

a . p l9. A methpd according to claim ,9, including the step 

of " " - ' ' ' ~\' 1 ■ ' % " ; 

20 varying said , proportion, of said video image, 

contributed , by said first depoded , output , in response to at least 
one of (a) User selection, (b) , formatting data received ni said input ; 
video data, and (c) pre-programmed processor instruction. 
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AMENDED CLAIMS 

[received by the International Bureau on 25 March 1998 (25,03.98); 
5 original claims 1-19 replaced by new claims 1-19 (5 pages)] 

1. Video; decoder apparatus for decoding input video data 
encoded in a plurality of encoding 1 formats, comprising: 

a first processor for decoding NffEG compatible data 
10 identified by a first data identifier to provide a first decoded video 
output; .> - ; - : - : ' x ' ' :r: : - : • • 

, : a second processor for decoding data encoded in ah Internet 
compatible data format, said Internet data Being identified by a second 
data identifier to provide a second decoded video output; and 
15 : a display ptocesisor • for- fofmatt^ second 

decoded videoi outputs for display as a comp>bsit8 vided ^nniage wherein 

the proportion of said video image contributed by said first' 
and second decoded outputs is variable and said first and second data 
identifiers are derived ^ from program said 
20 Internet data with video program said first 
: " : -decoded- video outpiiL - ' + ^%i.r-n » . * 

; c ^ 2. Apparatus- 4^° rail ig to Slaim % wherein 
. f r, v Isaid first and second processors • 'decode 1 "B&a separated using 
25 said first and second data identifiers^ frorfi a single composite input 
datastream. 

3/ Apparatus according to claim 1, wherein 
said first processor decompresses MPEG compatible 
30 compressed data, and 

said decompressed data is synchronized with second 
decoded video output using timing data in said MPEG compatible data. 

4. Apparatus according to claim 1, wherein 
35 said second decoded video output represents an index of 

web page information. 
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5. Apparatus according to claim 1, wherein 

. the format of said Internet compatible data includes data 
encoded in at least one of, a) TCP/IP format, b) HTML format, c) Java™ 
format, and d) ActiveX™ format., . 

6. Apparatus according to claim 1, wherein 

; said proportion of said, video image contributed .by said first 
decoded output is variable between 0 . and .100%. . , . ; : .■ ■ , 

...... : ^.tAPPVatus^ .according to ...claim- 1, wherein , ; ; : . <• 

■ :: , ? ai <* di? P^y ; P ro ?-? 4 s 'sor r:: stores said composite > video image irv a . 
pixel memory. " . . .. 

App.aratys according to. claim 1, wherein:. ; . i 

1 .said display prqcesson formats said first and second decoded 
outputs as separate images within said composite video; image and; j . vo-; 

said proportion of said composite video image contributed 
by said first and second decoded outputs is variable, in -response to at* 
least pre of .(a) User selection,, (b), formatting data : received -in ;said inputi 
Vlde °. ^ - and : ^ c !: P^i 0 ^ 1 !^ Pf?cessor instruction,,,. • .. 
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: 9. A method for decoding image representative input video 
data encoded in a plurality of encoding formats, comprising the steps of: 
'„ deriving first and second^ data identifiers from program map 
information / associating Internet data - with" video program content 
10 within said input video data; 

identifying MPEG compatible first image representative data 
using said: „ first data identifier; ; ^ - , " — : - s: 

- ; ; identifying second ; 5 image : r^ in an 

Internet compatible data format using said second data identifier; 
15 t '.r: -t\ ^decoding ;s;aid* identified- ^ ^irst in&ge ^epresientaltive data 

using a first MPEG data decoding method to provide a first " decoded 
output; 

-odeeoding $aid- ^identified :U second iitta>ge teprfeVenfative data 
usingti-^ second; Internet! data "decodihgp method to pnsvide a second 
20 decoded output; and > '*^' } '°' vr « "^*' " : 

formatting said first and second decoded outputs for display 
as a composite" videb im&gb whdirein "6f said Video image 

contributed by;isaid-^fir*t^ tend* Second 1 -dedoded ' oiitptf tt; J i!s^ymable." 

25 10. A method according to claim 9, further including the step 

of: 

t separating 1 Said ^rst- and E seconi ri imag^ PepTesentative data 
from a ; single : composite inpM datas^am. nc : ( ^ ir \ * ' " 

30 : II. tApparattis accbrdirig ^6 claim 9, including the stejp of ~ * 

determining whether a User is authorized to adcess said first 

image representative data and wherein 

said step of decoding said first image representative data 

occurs in response to said authorization. 
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12. Apparatus according to claim 11, including the step of 
decrypting said first image representative data in response 
to said User authorization. 
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13. A method according to claim 9, including the steps of 
receiving program guide information v and . .. . 
. receiving web page information; , for , display, -said^received 
web page information being selected ., from said .received program guide 
10 information. . " . ■ . r . 
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14. A method according to claim 9, including the; stepi of ; 

u ■ -receiying_. : .an index of. web- page r »n/prma*ion- '• Snd • including 

the step of 6 

r , fweiying r: .5aid second ..image^ jimemttfve- 'i/data^elected 
from, said index. ' ' , ... V vr , 



,,15, A .method ^cording... to^Iai^-^ tn in.cluding the,step :of 
: - , decom P res 4ng 1 MPEG compatible compressed , first: imaee 
20 representative data to provide said first decoded output '•"> — ; r - 



d * c ? din jS 5 - ec #:. C representative.,.-, iiaja- -formatted iin.at- 
least one of a) TCP/IP format, b) HTML format, c) Java™ format, and d) 
25 ActiveX™ format. ; . u . „ , ,' tc 

17 - A. - m ^ tn ,? d ai;cprding„to xlaim. 9, including ,the step of 
varying said proportion of said video , image ,] contributed- fry r 
said first decoded output in response to at least one of (a) User selection 
(b) formatting, data received in said input , video data,, ., and ;(c) pre- 
programmed processor instruction. •/ , . 
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^ _ 18. A methoci for decoding image representative input video 
data encoded in an Internet -compatible data format, comprising the 
steps of:- : \ r. 1 

S i identifying ....first Jmage. representative data encoded in an 
1 0 Internet cpjiipitible data format rising a -first data identifier; 

v ^decoding said identified first r imagec ' representative data 

using an ' Internet (lata decodi^r imethod:: to provide an index of web 
pageS informatibnT i^oeiated with:; video j gj&£ramc content in said input 
video datar ~i ; | Izi — ; ,/"£_...J ; / 

15 12 r " r , identifying second igniage representative data encoded in an 

Internet /Compatible dat$ ! format Rising a& second data identifier, said 
seconji ;image| representative* dataL.rppresenting a web page selected 
i from said- index of web • p*ge infOTn.atiori;V' .< *: 

r' ...\- ^ decoding sai4 [identified 'second^ ; image representative data 
20 using saicj Internet data decoding^method: fo provide said selected web 
page; and § • - ' \i ! ■ j ^ & £ C ■ 

^formatting said , selectedMveb pagfe * for; display. 

19. Ar method accojding -tp.. claim ; 18 9 including the step of 
25 deriving first: and second data ; identifiers from program 
map information ^associating ■ saidl selected v/efc page with said index of 
~ : web page information. • ! 



z & : < . 

7 v.w« .'. .» 
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